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Executive Summary 

In order to improve cost and schedule performance, the Discovery and New Frontiers 
(D&NF) Program Office studied life cycle cost (LCC) and schedule growth for five 
missions. The goal was to identify the underlying causes for the cost overruns and 
schedule delays, and to develop practical mitigations to assist D&NF projects in 
identifying potential risks and controlling the associated impacts to proposed mission 
costs and schedules. The study found nine systemic issues involving project management 
and systems engineering that were the primary contributors to LCC and schedule growth: 

1. Optimistic hardware/software inheritance and technology readiness assumptions 
caused cost and schedule growth in the detailed design and development phases 
(phases C/D) for all five missions studied. 

2. Insufficient project management and technical insight into contractor performance 
resulted in poor communications, schedule delays, and technical problems that 
were manifested as cost overruns in three missions. 

3. Inadequate planning for operations (phase E) resulted in significant LCC impacts 
in four missions. 

4. Mission replans were inadequate, and significant changes to mission scope, 
schedule, or funding profiles were not sufficiently understood or analyzed, 
resulting in unexpected cost increases and schedule delays in three missions. 

5. Projects demonstrated problems developing and maintaining Integrated Master 
Schedules (IMSs); problems stemming from inadequate IMSs were seen in four 
missions. 

6. The complexity drivers for fault protection and autonomy (FPA) capabilities are 
not well understood and projects underestimated the time and effort required to 
complete FPA development in four missions. 

7. Ineffective management structure and unclear roles and responsibilities resulted in 
cost and schedule impacts to three of the five missions studied. Issues resulting 
from the management structure compounded the effects of other project issues, 
such as overly optimistic heritage or technology assumptions, inadequate project 
schedules, and inadequate planning or replans. The management structure issues 
were exacerbated by project team member inexperience. 

8. Team players with limited experience in planetary mission development were a 
significant contributor to project management issues resulting in cost over-runs. 
Project team inexperience was a direct factor in only two of the missions studied, 
however it also contributed to other problems cited within this study. 

9. Many of the technical and project management drivers for LCC escapes were 
identified as issues (weaknesses, risks, concerns, or findings) during one or more 
of the mandated NASA project reviews, but there was insufficient follow-through 
to address (mitigate or refute) the panel’s conclusions and recommendations. 

The study also noted three observations: 
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1. The collection, analysis, and synthesis of the study data was much more intensive 
than anticipated, primarily due to the lack of official, formal program and project 
historical documentation. 

2. The D&NF programs, as well as NASA as a whole, need to address the credibility 
of project cost estimates, including independent cost evaluations. 

3. The majority of the underlying causes are embedded in the project approach 
during the concept study (phase A) and preliminary design (phase B) phases, but 
the actual cost or schedule impacts typically are not experienced until late in the 
development or operations phases (phases D and E). 

The most surprising result is that none of these findings are new. All of the findings have 
been reported over the years as project management and systems engineering issues; 
good project management and systems engineering practices have been identified 
addressing each of the findings. However, the study concludes that problems remain and 
additional attention is required to control or eliminate these problems and contain the 
resulting impacts to D&NF project costs and schedules. 


Background 

The Discovery Program (DP) is a science program of frequent, mid-class spacecraft 
missions that perform high-quality focused scientific investigations. Initiated in 1992, 
the DP was defined to ensure frequent access to space for planetary system(s) science 
investigations, emphasizing missions that can be accomplished under the leadership of 
the scientific research community. The DP comprises a long-term series of space science 
missions that are independent and uncoupled, but share a common funding and 
management structure. Since its inception, DP has successfully completed missions to 
study the Moon, inner planets, asteroids, comets, and solar wind. Missions or missions of 
opportunity currently in development or operations will continue exploration of the inner 
and outer planets, asteroids, comets, and the Moon. 

The New Frontiers Program (NFP) is a science program of medium-size spacecraft 
missions that perform high-quality focused scientific investigations. Initiated in 2003, 
the NFP was defined to pursue planetary missions that require resources beyond those 
available in the DP. The NFP comprises a long-term series of space science missions that 
are independent and uncoupled, but share a common funding and management structure. 
The NFP currently comprises two missions to study outer and dwarf planets. 

Both programs include two classes of projects: missions of opportunity (MOs) and full 
missions. MOs are characterized by being an element (for example, a science 
instrument) of another mission of any size not associated with either program, or by 
reusing existing NASA space assets in phase E. MOs have a not-to-exceed total cost to 
NASA as specified in the Announcement of Opportunity (AO), which at the time of this 
study was $35M. Given the smaller size and lower complexity of the effort, MOs were 
not included in the D&NF LCC study. 
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Full missions for both programs are selected through an open science competition and 
can include any science investigation involving solar system objects except for the Earth 
and the Sun, which are currently covered by other programs. D&NF missions are 
Principal Investigator (Pl)-led, complete, self-standing, and uncoupled Science Mission 
Directorate (SMD) investigations. The total cost to the National Aeronautics and Space 
Administration (NASA) for each mission has a not-to-exceed cost cap specified in the 
AO for the competition. At the time of this study, the Discovery mission cost cap was 
$425M and the New Frontiers mission cost cap was $700M. The mission cost cap covers 
the complete mission, including spacecraft development, mission operations, data 
analysis, and education and public outreach. Typically, the launch vehicle costs for past 
missions have been included in the mission cost cap, although the current trend for both 
programs is to exclude these costs from the project-controlled cap. 

D&NF missions are ultimately defined in terms of the science return from the mission. 
Level I requirements include the baseline science mission: the full set of scientific 
requirements identified for the mission, and the threshold science mission: the minimum 
set of science requirements below which the mission is not considered justifiable for the 
proposed cost. The delta between the baseline and threshold science requirements is used 
to define specific project de-scopes that can, with authorization from the program, be 
used to offset cost or schedule overruns. The use of de-scopes as a risk mitigation 
technique is built into the D&NF program approach to managing low-cost, cost-capped 
missions. 

Each PI is held responsible for proper execution of all aspects of the mission, including 
cost and schedule. Pis team with a NASA center, or an institution such as the Jet 
Propulsion Laboratory or the Johns Hopkins University Applied Physics Laboratory, to 
lead implementation of the mission. This home institution in turn assigns a project 
manager to lead the day-to-day activities. Development of one or more elements of the 
mission may be held in house at the home institution, or more often is delegated to a 
prime contractor, with additional subcontractors as required. Thus, the project 
management structure and lines of communication for each project include the PI, the 
Project Manager at the home institution, a project manager at a prime contractor, and any 
subcontractor management. 

The D&NF programs are managed by a single program office at Marshall Space Flight 
Center. The program office goal is to enhance the probability of mission success of 
D&NF projects through independent oversight and insight throughout all phases of the 
mission life cycle utilizing a knowledgeable and efficient team. From this perspective, 
success is defined as: 

• Delivering mission science to the PI (meeting the Level I requirements) 

• Ensuring the implementing organization’s success in delivering the spacecraft on 
cost and schedule (meeting the launch date and cost cap) 

• Meeting the program launch frequency requirement for science missions 
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Study Impetus & Approach 


As uncoupled, multi-mission programs emphasizing cost-capped Pi-led missions, the 
ability of the D&NF programs to meet their launch frequency requirements is driven by 
the ability of each individual project to meet its proposed and confirmed LCC and 
schedule. Looking at the missions over the history of the D&NF programs showed an 
increased frequency of cost overruns and schedule delays (see Figures 1 and 2). 

While there was a trend of increasing life cycle cost growth even in the earliest D&NF 
missions, the growth was contained in the early missions, showing an average cost 
growth of ~5% and a 
maximum of 15% for one 
mission. In contrast, the 
five recent missions 
selected for the study 
showed an average cost 
growth of 31%, with one 
mission reaching 50%.* 

Likewise, the average 
schedule growth for the 
D&NF missions increased 
from about 6% (20% for 
one mission) to 55%, with 
one missions showing a 
schedule growth of 153%. Note that schedule growth is shown in Figure 2 for only nine 
of the D&NF missions because the available documentation does not include the original 
proposed launch dates for the remaining two missions (see Observation 1 : Poor Program 
Documentation) 



Figure 1. D&NF Life Cycle Cost Growth 



Figure 2. D&NF Schedule Growth (Launch Date Slip) 


At a very high level, some 
of the cost and schedule 
growth was known to be 
attributed to changes in 
NASA philosophy 

regarding project risk, and 
the subsequent imposition 
of additional systems 
engineering and 

management processes that 
had not been included in 
the original mission 
proposals. However, the 


1 Study was executed prior to implementation of NASA guidelines to budget missions at the 70% 
confidence level. 
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trend has continued for missions that were proposed and accepted after this transitional 
period. 

Some of the cost growth may also be attributed to the selection of missions that are more 
complex and therefore missions that are inherently more difficult to accurately estimate 
than earlier missions implementing the “low hanging fruit”. However, these cost escapes 
directly affect the programs’ ability to fund new mission starts and meet their objectives 
to provide opportunities for the science community to propose and execute solar system 
exploration missions. Therefore, the D&NF Program Manager chartered a study to 
assess the cost escapes that have occurred on recent D&NF missions, determine how the 
cost escapes are making it through the institutional and D&NF evaluation and review 
processes, and determine what reasonable things could be done as a program to either 
prevent the cost escapes or manage them better. 

Study Objectives 

The objectives of the study were to: 

• Assess mission development process irrespective of organizational responsibility 

• Assess LCC growth at decision gates throughout the mission development process 

• Identify factors that contribute to the occurrence of unplanned costs and 
significant mission cost cap overruns 

• Provide specific recommendations and implementation plans based on the 
findings to improve current processes and provide a greater level of insight to 
make better-informed decisions throughout the mission life cycle 

Study Process 

To execute the study, five missions were selected from the two programs based on a 
recent history of exceeding proposed or confirmed costs. Those missions are highlighted 
in dark blue in Figure 1 and dark green in Figure 2. The selected missions covered a 
spectrum of complexity, cost growth, and maturity. All missions were well into or past 
the detailed design (phase C); two missions were already in operations, and one had 
completed its primary mission. 

The study process is illustrated in Figure 3. Decision gates (see Table 1) are mission 
milestones at which NASA makes decisions that authorize a project to proceed and 
commit funding to a project’s LCC. Each decision gate is associated with a detailed 
review of the project plans and proposed schedule and costs by a source evaluation board 
or a review board to ensure that the project can be executed within the proposed cost and 
schedule and to identify any risks against successful completion. The study looked at the 
reviews and products associated with each decision gate and compiled a preliminary 
timeline of the evolution of the project milestones and the total LCC for each mission. In 
addition to the decision gate data, the study looked at cost analysis data requirement 
(CADRe) results, milestone review products (Critical Design Review (CDR), System 
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Implementation Review (SIR), Operations Readiness Review (ORR), and Mission 
Readiness Review (MRR)), project status reports, and program budget exercises, in order 
to understand the drivers underlying the cost changes at each decision. 


_ Project 

E 

Rp Gate 
_J Analysis 




List of Major Findings 


Associated 

Recommendations 


Figure 3. D&NF Life Cycle Cost Study Process 


Table 1. Decision Gates 


Decision Gate 

Associated Products or Reviews 

Timeline Event 

Proposal Selection 

AO Proposal 

Phase A Start 

Project Selection 

Concept Study Report (CSR) 

Phase B Start 

Project 

Confirmation 

Preliminary Design Review (PDR); 
Confirmation Review (CR) 

Project Confirmation; 
Phase C Start 


Members of the study team then met in person with program and project representatives 
to develop an accepted historical timeline of cost increases and schedule changes over the 
life cycle of each selected mission. Program representatives included the program office 
mission managers (MMs) and Headquarters (HQ) program executives (PEs) assigned to 
the missions selected for study, budget analysts, and representatives from the Technical, 
Management, Cost, and Other (TMCO) source selection process. Project representatives 
included mission Pis, Project Managers, budget analysts, and representatives of the home 
institution. 
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Considerable effort was required to reconcile the cost history with the underlying drivers. 
In no case did a complete documented history of the cost and schedule changes exist for a 
mission. Instead, the cost and schedule changes had to be reconstructed from an 
inconsistent and sometimes contradictory set of documents held at the project, program, 
and HQ levels, as well as the memory of members of the project teams and program 
management. The results of this analysis were documented in a consistent set of products 
for each mission: 

• A detailed spreadsheet summarizing the discussions with the program and project 
representatives 

• A project development timeline identifying the original life cycle phase 
boundaries, and any changes to the proposed major milestones 

• A cost history breakdown, including the factors contributing to the cost increases 
at each decision gate 

• A narrative summary of the driving changes to the mission cost and schedule of 
the full mission life cycle 

Note that during this process, the study focus was on the primary cost and schedule 
drivers and their relative potential for impact, not the pinpoint accuracy of the associated 
cost estimates. As will be discussed later, it was very difficult to trace the detailed cost 
and schedule history for each of the missions. Even after discussions with the project, it 
was often impossible, given the passage of time, to accurately break out individual costs 
between multiple items included in a single line in a budget submit. However, while 
frustrating, these were not found to invalidate the study approach nor to interfere with the 
identification of the primary drivers for the cost and schedule impacts. 

Finally, the detailed mission timelines and cost breakdowns were used to identify the 
overarching drivers for project cost overruns and schedule delays. In general, the drivers 
affected multiple missions. However, the final set of findings also included drivers that 
resulted in a significant cost or schedule impact for only one or two missions, if they 
represented a significant potential risk to future D&NF missions. Also, the emphasis of 
the study was on cost and schedule risks that could be directly affected by D&NF 
program or project management. Thus, the findings do not include cost drivers that were 
outside the control of the project or program. Cost or schedule drivers resulting from 
NASA policy or budget decisions are noted in the data, but are included in the study 
results only as they contribute to the nine findings of the study. 
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Results 


The final trace of the cost growth from the CSR (start of phase B) across the five 
missions studied is shown in Figure 4. The percent growth is shown for three data 
points: the end of preliminary design at the PDR, the subsequent CR, and the final 
project estimate at the time of the study. The final project cost growth shown in Figure 4 



Figure 4. Recent Mission Life Cycle Cost Growth 


may not represent the total project LCC, as only one of the missions studied had 
completed its primary science mission at the time of the study. As seen in other cost 
growth studies, the largest cost growth in all cases occurred after project confirmation. 


Figure 5 traces the schedule growth in terms of launch delays from the dates proposed in 
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Figure 5. Recent Mission Project Schedule Growth (Launch Date Slip) 


the CSR for the five missions studied. Similar to the project cost growth, the schedule 
shown for three data points: the end of preliminary design at the PDR, the subsequent 
CR, and the actual or proposed launch date at the time of the study. Thus, the schedule 
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growth does not include increases to the overall mission duration due to changes in 
encounter (e.g., fly-by) or orbit insertion date resulting from a launch delay. The 
accompanying increases in operations (phase E) duration ranged up to two years for the 
missions studied. It should also be noted that at the time of the study, one mission 
(mission 3, indicated by an asterisk in Figure 5) had not launched, so the schedule 
growth shown may not represent the full launch delay experienced by the project. 

As expected, not all cost and schedule growth could be traced to factors under the control 
of the program or the project. Several projects experienced cost growth when additional 
NASA requirements that were not costed in the original mission proposal were passed on 
to the projects. Some examples of these included recommendations from the NASA 
Integrated Action Team (NIAT), increasing reserves to a mandated level, or 
implementation of full cost accounting within NASA. In addition, three projects 
experienced both cost and schedule growth as the result of a stand-down imposed by 
NASA due to budget issues. Although noted as cost drivers during the study, these were 
not included in the findings in order to concentrate on the issues within the control of the 
program or the project. 

Major Findings 

The study returned nine major findings covering multiple aspects of program and project 
management, and project systems engineering (see Table 2) that are under the control of 
the program or project. There is no implied ranking to the order of the findings based on 
potential impact to a specific mission, or the number of potential missions experiencing 
an impact. 

Table 2. Major Findings 


1 

Heritage and Technology Assumptions 

2 

Insufficient Project Insight 

3 

Inadequate Planning for Operations 

4 

Inadequate Mission Replans 

5 

Integrated Project Schedules 

6 

Fault Protection and Autonomy 

7 

Ineffective Management Structure 

8 

Project Team Inexperience 

9 

Consideration of Review Team Findings 


The most surprising result of this study is that none of these findings are new. All of 
these have been identified by practitioners and researchers as areas of project 
management or system engineering risk. Institutional standards or best practices have 
been developed in these areas; all are routinely covered in project management or system 
engineering training courses. However, the study indicates that management and 
engineering processes in these areas are not yet fully mature nor consistently and 
routinely implemented across NASA projects. Despite the years of experience in 
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executing spacecraft missions, the development of institutional standards, and the 
implementation of training programs, NASA projects still experience cost overruns 
resulting from basic project management and systems engineering issues. 

Finding 1: Heritage and Technology Assumptions 

Optimistic hardware/software inheritance and technology readiness assumptions caused 
cost and schedule growth during detailed design (phase C) and development (phase D) 
for all five missions studied. Missions frequently assume the use of advanced technology 
systems or heritage in their proposals to enable a mission or reduce the overall mission 
risk and cost. However, the missions studied failed to achieve the proposed reduction in 
cost and/or schedule duration, and all experienced significant cost and schedule growth 
during the development and test. 

The cost and schedule growth was not found to be the result of technical issues. 
Institutional inheritance and technology readiness processes appear to adequately prevent 
technical issues from impacting mission operational success. However, a combination of 
overly optimistic assumptions regarding technology readiness or the potential savings 
associated with inheritance and institutional inheritance and technology readiness 
processes allowed cost/schedule escapes to impact phases C/D. In general, the overruns 
could be traced back to: 

• Inadequate understanding of the heritage system’s performance within the 
proposed project design or mission environment 

• Project personnel with insufficient experience with the heritage system 

• Inadequate scoping of impacts to implement a new technology for space 
flight 

During the proposal development, projects assume the use of various instruments, 
sensors, or spacecraft buses and cite heritage from previous missions as the basis for cost 
or schedule savings. However, typically little or no allowance is made for structural, 
mechanical, electrical, or thermal redesign required to use the item within the constraints 
of the new spacecraft. Studies to mitigate common risks such as internal electrostatic 
discharge (IESD), cross-talk, or thermal stability may not be included in early design 
costs for heritage hardware. Likewise, insufficient analysis is performed in formulation 
(phases A and B) to accurately determine redesign that might be necessary or identify 
additional testing for recertification of heritage hardware for use in a new thermal or 
radiation environment. 

In addition, the schedule and cost savings achievable from the use of heritage hardware 
are highly dependent on the extent to which the personnel working on the spacecraft have 
previous experience with the heritage hardware. The study data showed that projects 
failed to sufficiently account for the learning curve required to bring an inexperienced 
team up to speed on heritage hardware. The impact resulting from a lack of accurate, up- 
to-date documentation on heritage hardware also was not identified or adequately 
compensated for in schedules and cost estimates. 
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Similarly, project estimates of the effort required to incorporate a new technology into the 
space system were under-scoped. Programmatic issues, such as launch approval for 
nuclear systems, were not adequately understood and therefore were underestimated. 
Ancillary support, such as required equipment for development and ground testing were 
inadequately defined in early design and therefore were inadequately covered in the 
proposed mission costs. Finally, impacts to other spacecraft systems (e.g., additional 
shielding for other spacecraft systems after the introduction of a nuclear component) may 
not be identified during proposal development or preliminary design and therefore not 
included in the proposed and confirmed costs for the mission. 

Project management needs to realistically assess both the benefits and risks associated 
with heritage and new technology, and ensure appropriate mitigations are identified and 
implemented. Proposals and early design products should clearly demonstrate the 
relationship between heritage/technology rankings and cost/reserve estimates. This 
relationship should form the basis of a living reference between heritage/technology 
rankings and cost/reserve estimates, updated throughout formulation and implementation. 
High risk heritage/technology assumptions should be captured as project risks, tracked 
against mitigation actions, and assigned cost liens or threats that are updated at each 
project review. 

Standing Review Board (SRB) or review team members need to be selected to ensure that 
members possess current expertise in applicable areas, including not just the heritage or 
new technology but the operational environments encountered during the mission. 
Heritage and technology assumptions, proposed and completed mitigation activities, and 
the associated cost liens or threats need to be independently and rigorously reviewed at 
each major milestone. 

Finding 2: Insufficient Project Insight 

Insufficient project management and technical insight into contractor performance 
resulted in poor communications, schedule delays, and technical problems that 
manifested as cost overruns. This was a significant contributor to cost overruns on three 
of the five missions studied. 

As uncoupled programs of independent, Pi-led missions, the D&NF programs rely on the 
processes and procedures of the home institution to meet NASA-imposed standards. 
Cost overruns occurred when insufficient insight or oversight by the project resulted in 
contractor specifications, processes, and procedures that did not meet the home 
institution’s expectations. This can result from a new teaming relationship, perhaps 
created to provide an unusual mission-enabling technology or to support reuse of a 
technology in a new or different manner. This can also result from transfers between 
business units of project personnel with significant experience in other methodologies, 
but with insufficient training in NASA’s standards, processes, and procedures. In each 
case, organization “cultural” differences and insufficient flow down of detailed 
requirements led to products that did not meet institutional standards. 
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In some cases, the result was “gaps” in processes, where each side was expecting the 
other to routinely perform specific activities (e.g., test). In other cases, the result was a 
part that could not be certified for use according to NASA standards. In turn, the process 
gaps and uncertified products led to rework and retest of both hardware and software 
before the product could be accepted. In addition, waivers were sometimes required and 
approved in order to limit cost and schedule increases, resulting in an increase to the 
overall mission risk accepted by project and NASA. Cost overruns also resulted from 
hardware mishaps stemming from processes that did not meet NASA standards, as well 
as from adding personnel to oversee a process to mitigate the risk of a mishap during a 
critical process. 

During the early formulation of a project, all parties need to work together to clearly 
communicate requirements and standards down the organization chain to all levels, 
including contractors and subcontractors. Likewise, institution or corporate practices 
need to be clearly and concisely communicated up the organization chain. Even with this 
communication, the parent organization(s) need to implement a sufficient level of insight 
to ensure that the communication does occur, that all levels are in agreement, and that the 
delivered products will meet home institution requirements and standards. This is critical 
for all teaming arrangements, even organizations that have a long and very successful 
teaming relationship on multiple projects, as changes in personnel can introduce 
differences in expectations. 

Finding 3: Inadequate Planning for Operations 

Mission selection, review, and management processes are heavily focused on hardware 
development schedules and costs. Operations costs are typically small compared to the 
spacecraft development and test costs. This, combined with the long lead time for 
realizing phase E costs, can lead to de-emphasis of phase E support requirements during 
proposal and preliminary design cost exercises. However, even moderate yearly 
underestimates can present significant LCC impacts for missions with long phase E 
durations, becoming a significant contributor to cost overruns on four of the five missions 
studied. 

Underestimation of the proposed mission operations costs was seen to result from: 

• Underestimating the complexity of mission operations 

• Inadequate planning for sustaining engineering 

• Ignoring special requirements for long duration missions, e.g., knowledge 
retention, software and hardware refresh, technology evolution, institutional 
staffing considerations 

An additional factor, underestimating the ground system support requirements leading to 
delaying development of operational capabilities into phase E was also a significant 
contributor to cost increases during this time frame. 
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Insufficient resources and effort are assigned to defining an operations concept and 
estimating operations costs during the proposal and phase B activities. Missions are often 
proposed, staffed, and estimated as “routine operations,” ignoring drivers in the 
spacecraft and mission design that require additional ground system tool capabilities or 
additional operational staffing support. In addition, changes to the spacecraft design are 
not always adequately assessed for their effect on operations, particularly increases in 
operational complexity, as part of system engineering trade studies. This could be the 
result of processes that failed to include the full LCC impact during system design trade 
studies, insufficient information available to the operations team to allow a realistic 
assessment of the effect on operations, or insufficient experience in the operations team 
to understand the full ramifications of a spacecraft design change. 

Even for straightforward missions, phase E costs can be underestimated due to 
inadequate planning for sustaining engineering. Overly optimistic assumptions about 
operational savings due to lessons learned from previous projects are often not realized 
due to spacecraft “idiosyncrasies” that are not identified until after launch. Overly 
optimistic assumptions are also made about the need for flight software maintenance after 
launch or general engineering support during cruise phase for monitoring spacecraft 
health and safety or responding to spacecraft anomalies. 

In addition, projects typically ignore one or more of the special requirements for long 
duration missions. For deep space missions, the requirement for minimal staffing during 
cruise phase in order to control total phase E costs competes with the need to retain 
critical staff knowledge and a skill base to support engineering activities during critical 
milestones such as planet flybys, as well as science operations at the science target. The 
staffing plan is then found to be unworkable during detailed planning for engineering 
activities before launch, or when the detailed science planning begins during cruise. 
Likewise, special considerations applying to long duration deep space missions, such as 
routine software and hardware refresh and the incorporation of evolving technologies, are 
inadequately represented in the confirmed mission costs. While it might be the initial 
intent of the project to control costs by rejecting hardware or operating system upgrades, 
this may be found to be unworkable when vendors suspend maintenance on certain 
hardware or software product lines, requiring projects to upgrade or face unsupported 
hardware failures. 

Finally, inadequate planning for the development of operational tools often results in 
software development being delayed in phase E. This is most common for fault 
management software (see Finding 6: Fault Protection and Autonomy) and science 

operations tools, but was seen to occur in many areas, including flight software, mission 
design, scheduling, and spacecraft analysis tools. While the actual development should 
be covered by unspent phase C/D funding held over into phase E, in practice this is rarely 
sufficient. Not only was work delayed into phase E, but staffing was increased during 
phase C/D causing most, if not all, of the planned development costs to be spent in phase 
C/D, and requiring additional funding for development during phase E. In addition, the 
unexpected development typically levies additional effort on the operations teams for 
testing, parallel (shadow) operations, reprocessing, and, potentially, procedure and 
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command product development. This additional work is rarely covered by the original 
phase E costs estimated during formulation. 

Project management should emphasize the need for a full and accurate LCC estimate 
from the earliest stages of proposal development. This should include a phase E staffing 
estimate by primary operations functional area that can clearly be traced to and supported 
by the proposed operations concept. Early design activities need to emphasize the 
importance of an increasingly detailed operations concept developed in parallel with the 
spacecraft that identifies, for example, plans for equipment maintenance/replacement and 
parts obsolescence, and longevity plans for retaining/replacing key personnel. 

If circumstances force a project replan, project management needs to reassess key 
elements of the operations concept and consider the potential impact to mission life time, 
complexity, and operations staffing. In the event that significant phase D work must be 
deferred into phase E, project management should rigorously assess the impact on 
operations personnel and manage the deferred phase D work as a formal development 
effort, including all the appropriate schedules and metrics. 

Review teams should independently assess the operations staffing and cost estimates and 
the phase E reserve plans against the mission operations concept. The Confirmation 
Review out-brief should specifically address the operations concept and the assessment 
of the project phase E plan, including any potential risks to the plan based on the 
operations concept. 

Finding 4: Inadequate Mission Replans 

The impact of significant changes to mission scope, schedule, or funding profiles were 
not sufficiently understood, resulting in unexpected cost increases and schedule delays. 
Not all replans were initiated as the direct result of project issues; several of the projects 
studied experienced delays or replans as the result of NASA-initiated changes. Other 
project replans were the result of project issues stemming from one or more of the 
findings identified in this study. In either case, project replans are often time constrained 
in order to maintain project schedules or meet NASA budget cycle deadlines. 
Insufficient analysis of the full implications of the changes to the total mission life cycle 
resulted in additional unanticipated cost and schedule growth in three of the five missions 
studied. 

For NASA-driven changes (excluding directed launch delays), the most common cause 
for an inadequate replan was a lack of understanding of the change to the scope or 
process requirements. As one example, the system engineering and test processes 
resulting from additional risk reduction activities (in response to NIAT 
recommendations) were not well understood at the time of the original replan assessment. 
As the full impact on project resources became apparent, the project was forced to replan, 
adding additional resources (cost) to meet schedule commitments. 

Similar to the issues noted in Finding 3, above, impacts to phase E resources were 
typically missed or underestimated in replans that resulted in launch delays. The realities 
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of deep space missions are often brushed over given the time available to replan, 
assuming (correctly) that the spacecraft can still be operated. However, in one case a 3 
month launch delay added 2 years to the operational lifetime, as well as increasing the 
number of critical activities by ~50%. The total effect on operations of the overall 
increase in mission complexity, including the density of critical activities, was not fully 
realized until two years into phase E. 

Likewise, effects of a launch delay on heritage hardware or technology assumptions are 
often not completely analyzed during a replan. A launch delay can change the 
assumptions under which the original environmental analyses were performed (e.g., 
mission delayed to solar maximum). A new mission design resulting from the launch 
delay can completely alter the spacecraft thermal environment or the mission lifetime. 
The impacts from these types of issues are often not fully realized until the replan is 
complete and approved, and the engineering team revisits the original analyses. 

Finally, any replan that defers phase C/D work (software and ground segment hardware) 
into phase E, needs to be carefully assessed to ensure that all operational considerations 
have been included. This includes not only the development work and any associated 
ground or in-flight testing, but also interim operational support for workarounds; 
hardware and personnel requirements for training, scenario tests, and data reprocessing; 
and documentation updates. 

In order to ensure an accurate and complete mission replan, the program and project need 
to work together to clearly define the goals and scope of the replan and ensure sufficient 
time is allocated to perform a detailed assessment. Projects need to carefully consider 
ancillary effects such as heritage assumptions (see Finding 1: Heritage and Technology 
Assumptions), as well as specifically address any operational considerations (see Finding 
3: Inadequate Planning for Operations). The project should then develop a detailed 
revised baseline, including not only the cost and schedule impacts but also the underlying 
assumptions and any risks associated with the revised baseline. The replan should be 
confirmed by a complete and independent evaluation of the assumptions and risks and the 
resulting impacts to project LCC and schedule. In the event that time constraints do not 
permit a full evaluation prior to submitting revised cost and schedule data, the full 
process including independent review still should be completed as soon as possible after 
submitting the revised data. 

Finding 5: Integrated Project Schedules 

The lack of a comprehensive, integrated project schedule resulted in uncoordinated 
activities, inefficiencies in resource management, and increased costs. A valid, 
comprehensive IMS is required for earned value management (EVM), which is an 
accepted and required tool for measuring project progress and controlling project costs. 
Yet, projects continue to experience problems developing and maintaining IMSs. 
Schedule delays and the resulting cost overruns stemming from inadequate IMSs were 
seen in 4 out of 5 of the missions studied. 
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In the missions studied, every home institution required and set standards for an IMS, and 
every project had implemented one. In spite of this, the study identified IMS 
inadequacies that could be directly traced to cost overruns and schedule delays, 
including: 

• Missing critical milestones and major events 

• Missing logical relationships (interdependencies), or unidentified or incomplete 
critical paths 

• Over-specified and unmanageably large number of tasks 

• Multiple separate, uncoordinated schedules 

The lack of a comprehensive, usable IMS resulted in uncoordinated activities, 
inefficiencies in resource management, late identification of risks to the critical path, and 
increased costs. 

One of the most common problems relating to IMS inadequacies over the missions 
studied was missing critical milestones and major events. In at least one case, events 
were missing even though the IMS was extremely detailed, extending to a low level of 
detail in all areas. In fact, the very size of the IMS was a likely contributor to the missing 
milestones and events, in that it was too large for a meaningful review of all the events 
and relationships. The missing events resulted in resources being underestimated for 
completing critical tasks, and meant that management had insufficient or confusing data 
for tracking performance across the project. 

Most IMSs also had missing logical relationships (interdependencies) or unidentified or 
incomplete critical paths. These were a common cause for underestimating resources, 
poor decision-making, and resulting schedule delays, with the added expense of 
additional resources for replans. 

Finally, both of the above issues were compounded by the existence of multiple separate, 
uncoordinated schedules that needed to be rolled together to understand the project status, 
rather than one true IMS. Although considerable effort was expended by projects on 
merging, or building automated processes to merge, the schedules, the mere existence of 
multiple independent and uncoordinated schedules compounded the difficulties in 
reviewing the schedule and ensuring that all critical milestones, major events, logical 
relationship, and critical paths had been identified and were being tracked. This in turn 
increased the likelihood of problems in tracking performance and the early identification 
of risks to the critical path. 

The importance of a usable IMS as a routine project management tool cannot be over 
emphasized. Project management must ensure that a valid, comprehensive IMS is in 
place and forms the basis of and facilitates EVM reporting. More importantly, project 
management must ensure that the IMS can be used easily as a routine part of monthly, 
weekly, and, at times, daily assessment of project performance and status across all 
elements and levels of the project. 


D&NF Life Cycle Cost Study Report 


Page 18 



The Integrated Baseline Review (IBR) should be used to confirm the content of the 
schedule and the processes for maintaining the schedule, as well as the ability of the 
schedule to support weekly and monthly planning and annual reviews. The actual 
integrated schedule and associated processes should be reviewed as part of project 
milestone reviews (PDR, CDR, and SIR) and periodically throughout the life cycle. 

Finding 6: Fault Protection and Autonomy 

The study also found that missions underestimate the time and effort required to complete 
FPA development. Four of the five missions studied experienced cost overruns, schedule 
delays, and even launch delays in order to complete the required FPA capabilities. 

Underestimation of FPA complexity resulted from difficulties: 

• Defining the appropriate autonomy requirements and the proper level of fault 
protection 

• Evaluating overall FPA complexity, including all hardware, software, and 
operations components 

• Estimating test requirements, including hardware and software fidelity and test 
coverage 

• Estimating required resources, including subsystem and operations personnel 
support, hardware simulation facilities (test beds) 

In general, FPA requirements tend to be defined late in development. This means that the 
full complexity of the FPA subsystem is not well understood during formulation, and the 
resources required for FPA development, test, and execution are not completely covered 
in the cost estimates prior to confirmation. In addition, FPA may not be clearly defined 
as an distinct engineering discipline. Instead, it is often included as an added 
responsibility of the system engineering team, and given a lower priority than critical 
systems engineering issues involving power or mass margins. Even when FPA is staffed 
as a separate engineering discipline, the staffing level may not be sufficient to allow the 
engineer(s) to follow the hardware and software trades and decisions closely enough to 
understand their impact on FPA complexity. This limits the project’s ability to mitigate 
increasing FPA complexity by appropriate system trades. 

Initial underestimation of FPA requirements and complexity, coupled with the growth in 
FPA complexity during development, requires projects to add additional resources to 
FPA development and testing in phase D in an attempt to maintain schedules. However, 
this may not be sufficient, and launch constraints can result in development work being 
held over to phase E. Not only does the delayed FPA work represent an impact to phase 
E funding, but it typically also requires additional operations personnel support for 
workarounds and changes to databases, documentation, and procedures while the FPA 
development is completed. The need for additional support can also be seen in the 
subsystem and system engineering areas, as additional subject matter experts are required 
to deal with analysis and anomalies until the FPA capabilities are complete and tested 
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operationally. Incomplete implementation at launch also represents an additional risk to 
mission success until the FPA capabilities can be completed and fully tested in flight. 

NASA as a whole, including the program, institutions, and projects need to develop a 
better understanding of the detailed cost drivers associated with FPA, in order to improve 
the initial estimate of the resources required. 2 Project management needs to ensure that 
FPA is treated as a separate and equal engineering discipline, with the resources and 
processes required to ensure that FPA complexity is considered as a factor in system 
trades and design decisions. Finally, project management needs to ensure that FPA 
development and testing is incorporated into the master schedule at a sufficient level to 
allow management to accurately track progress and plan for changes in resource 
requirements before issues reach a critical mass. 

Finding 7: Ineffective Management Structure 

Ineffective management structure and unclear roles and responsibilities resulted in cost 
and schedule impacts to three of the five missions studied. Issues with management 
structure occurred primarily within projects involving multiple organizations, and were 
exacerbated by project team member inexperience (see Finding 8: Project Team 

Inexperience). Ineffective management structure also served to compound the effects of 
other embedded project issues such as overly optimistic heritage or technology 
assumptions (see Finding 1: Heritage and Technology Assumptions), inadequate project 
schedules (see Finding 5: Integrated Project Schedules), and inadequate planning or 

replans (see Finding 3: Inadequate Planning for Operations and Finding 4: Inadequate 
Mission Replans). 

The problem areas noted during the study included: 

• Inconsistent project reporting and decision-making 

• Unclear lines of technical authority 

• Unconnected systems engineering across multiple organizations 

• Unclear responsibility for system integration 

In some cases, these resulted in changes to scope and commitment of reserves without 
adequate review and appropriate controls. In some cases, they resulted in “gaps” in 
definition and coverage of critical tasks. While in others, they resulted in a lack of clear 
authority and accountability for decisions and activities. 

During the early formulation of a project, all parties need to work together to clearly 
define and communicate roles and responsibilities to all levels, including contractors and 


2 Since this study, the D&NF Program Office, on behalf of NASA’s Planetary Science Division, sponsored 
a Fault Management Workshop to bring subject matter experts from across NASA together and begin to 
understand the issues and drivers affecting FPA cost overruns. The results from this workshop are 
documented in the report: Spacecraft Fault Management Workshop Results for the Science Mission 

Directorate, Planetary Sciences Division. 
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subcontractors. This definition needs to include clear paths for communication, even 
among teams with experience working together. Based on their experiences, review 
teams should specifically assess the project’s: 

• Management structure for clear reporting and decision authority 

• System engineering and integration responsibilities for gaps and overlap 

• Technical authority structure 

In addition, they should assess the team understanding of and the on-going effectiveness 
of the management structure. 

Finding 8: Project Team Inexperience 

Teams that include major players with limited experience in planetary mission 
development are a major contributor to project management issues resulting in cost over- 
runs. Project team inexperience was a direct factor in only two of the missions studied; 
however, it contributed to problems in a number of other areas cited within this study. 
The inexperience was manifested in many ways: 

• Complex or poorly-defined management structure, roles and responsibilities, and 
communications 

• Inadequate development schedules and implementation of performance measuring 
techniques 

• Inadequate performance oversight and configuration management: institute to 
prime and prime to subs 

• Incorrect or inadequate heritage or technology evaluations 

• Inaccurate cost estimates, inadequate cost control and management of reserves 

Given there may be strategic benefits to selecting “new” team members, the project and 
program should assume a greater cost risk and operate with increased insight/oversight 
for less experienced team members. Both projects and the review teams should: 

• Assess the relevant experience for major partners and identify associated risks 

• Develop mitigation plans, including mentoring and active sharing of lessons 
learned from more experienced partners 

• Identify and carry a cost threat, adjusted to reflect continuing team performance 

Review teams should independently assess the overall effectiveness of the management 
structure and any mitigation activities at the major milestone reviews. 

Finding 9: Consideration of Review Team Findings 

One of the more surprising findings of the study was that many of the technical and 
project management drivers for LCC escapes were identified as issues (weaknesses, risks, 
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concerns, or findings) during one or more of the mandated NASA project reviews. In 
other words, NASA commissions senior-level expert review panels, but does not always 
follow through and address (mitigate or refute) the panel’s conclusions and 
recommendations . 

There was little evidence indicating a consistent approach to responding to the findings 
and recommendations of a review team. At the time of the study, there was little 
evidence of formal documentation of findings and recommendations, and there was no 
indication that risks, budget adjustments, or cost threats were created to follow through 
on the board results. In many instances, the issues identified by a review team but not 
addressed, significantly affected a project’s cost and schedule and the program budget. 

The changes to the formation of SRBs implemented in NPR 7120.5D, NASA Space Flight 
Program and Project Management Requirements, are one step toward providing the 
continuity to ensure issues raised by review teams are addressed. However, project 
managers must allocate sufficient time and resources as part of the review schedule to 
seriously consider, analyze, and respond to issues raised by review panels. For each 
issue, project managers should carefully trade the cost of early mitigation versus the 
potential cost if the issue worsens or is realized (see Observation 3: Embedding vs. 
Realization of Impacts). If no immediate action is taken to resolve or mitigate each issue, 
the project should create and track a risk and associated threat to the project reserves to 
maintain visibility into the issue by management as the project matures. 

Observations 

In addition to the findings discussed above, the study returned three observations 
covering issues that are primarily programmatic in nature (see Table 3). There is no 
implied ranking to the observations. These observations were seen during the analysis of 
the mission data for all of the missions studied. 


Table 3. Observations 


1 

Poor Program Documentation 

2 

Cost Credibility 

3 

Embedding versus Realization of Impacts 


Observation 1: Poor Program Documentation 

The collection, analysis, and synthesis of the study data were much more intensive than 
anticipated. This was primarily due to the lack of official, formal program and project 
documentation. There was little traceability of cost or schedule impacts to program-level 
decisions or direction (e.g., official letters, documents). Cost or schedule changes were 
often “grouped” into high-level categories, making it difficult to trace the changes to 
detailed cost drivers. Records of year-to-year or life cycle-phased cost commitments and 
obligations to projects were often inconsistent and different players (project, program 
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office, HQ) disagreed on the details of the project timelines, cost increases, and scope 
changes. 

Given that the study concentrated on identifying the largest drivers, the data were still 
valid for identifying driving issues. However, programs and the projects need to be 
diligent in documenting decisions and project direction. Significant programmatic 
direction to a project should be documented in a NASA memorandum. Project budget 
adjustments should be documented with sufficient detail to understand the cause and any 
effects of the change; this should include documented feedback to projects regarding the 
results of the planning, programming, budgeting, and execution (PPBE) process. Finally, 
significant project meeting decisions should be documented in official minutes 
(management council, Program Control Board (PCB), etc.) that are sent to key program 
and project attendees. 

Observation 2: Cost Credibility 

Both D&NF programs, as well as NASA as a whole, need to address the credibility of 
project cost estimates, including independent cost evaluations. Even when technical and 
programmatic risks were identified early, the associated cost estimates did not adequately 
cover the identified risk. In many cases, more than one cost increase was seen for a given 
technical or programmatic issue, due to inadequately scoping the costs to analyze and 
resolve an issue or implement a new NASA-required process. In addition, even though 
reviewed by technical, scheduling, and cost experts, the underestimates were not 
identified and therefore were subsequently approved by NASA management. As noted 
above, in some cases this was directly related to the time allocated for a replan. 
However, in other cases, there was no observable correlation to a tight replan schedule. 
Rather, the cost increases appeared to result from inadequate understanding of the issues, 
required processes, and potential impacts on the mission. 

Observation 3: Embedding versus Realization of Impacts 

The third observation addresses an overarching issue affecting all the missions studied. It 
is predicted to be a fundamental issue that needs to be addressed in any plan to mitigate 
the findings identified by this study. 

Figure 6 shows the timing associated with the cost and schedule impacts addressed by 
the findings of this study. The time period during which each issue was embedded into 
the assumptions or design of the appropriate missions is shown in blue on the diagram, 
while the time period during which each issues was realized as an actual cost impact or 
schedule change is identified in red. With one exception, the impacts associated with 
replanning (Finding 4: Inadequate Mission Replans), all impacts are embedded (but not 
yet recognized) within the project cost and schedule plans by project confirmation at the 
end of phase B. However, the majority of the cost and schedule impacts are not realized 
as a request for increased project funding or a replan of the project schedule until well 
into phase D. 
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Figure 6. Embedding versus Realization of Cost & Schedule Impacts 

At the same time, the ability to descope content in order to control project cost falls 
sharply after the end of phase B. This is primarily due to the maturity of hardware and 
software designs, which are at a sufficient level that the cost of redesign to omit a piece 
of hardware, or select an alternative, equals or exceeds the savings associated with the 
descope. Thus, although both of the D&NF Programs require descopes as mitigation for 
the inherent risk in low-cost, cost-capped missions, descopes in and of themselves do not 
provide sufficient protection against the risk of schedule replans and cost overruns 
embedded within the project plans. 


Summary 

The D&NF Program Office LCC Management Study provides a detailed look at the 
drivers underlying cost overruns and schedule delays for five D&NF missions. While 
none of the findings are new, the study underlines the importance of continued emphasis 
on sound project management techniques: a clean project management structure with a 
clear definition of roles and responsibilities across the various partners in a project, an 
understanding of institutional standards and procedures and any differences among the 
partners, and the critical need for a comprehensive IMS that can be used easily and 
routinely to identify potential threats to the critical path. The study also highlights the 
continuing need for realistic estimates of the total LCC. Sufficient time and resources 
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must be allocated early in a project to ensure that the appropriate trade studies and 
analyses are performed across all aspects of a mission: spacecraft, ground system, 

operations concept, and fault management, to ensure that proposed and confirmed costs 
truly reflect the resource requirements over the entire mission life cycle. These studies 
need to include a realistic review of the assumptions underlying the use of new 
technologies, the integration of heritage and new hardware and software into the total 
mission environment, and any development and test savings based on heritage technology 
and lessons learned. Finally, the LCC Management Study stresses the need to listen to, 
carefully consider, and take positive action regarding the issues raised during reviews by 
the expert review teams. 
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Appendix A. Terminology 


Autonomy; The ability of the spacecraft to operate without intervention from ground 
operations. Together with fault protection, it defines the capability of a spacecraft to 
execute a critical science activity in deep space (e.g., encounter) without real-time 
intervention. 

Baseline Science Requirements : That mission which, if fully implemented, 

accomplishes the entire set of scientific objectives identified at the initiation of the 
mission. 

Confirmation : Authorization of a project to begin implementation; the culmination of 
PDR, CR, and Key Decision Point C (KDP-C). 

Decision Gates: Mission milestones at which NASA reviews project plans and makes 
decisions that commit funding to a project’s LCC. The decision gates used for this study 
were the selection of proposals (start of phase A), selection of a project for development 
(start of phase B), PDR/CR (start of phase C). 
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Fault Protection: The detection of and response to in-flight anomalies. The response 
may be “layered,” some occurring autonomously onboard, others requiring intervention 
from the ground. 
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Heritage Systems: Hardware, software, and procedures with previous flight history that 
are reused for a new mission in order to enable a mission capability or reduce overall 
mission cost, schedule, or risk. 

Inheritance: The process of evaluating the compatibility and benefits of heritage 

systems to the requirements of a new project, and validating the level of reuse or rework 
(design, fabrication or coding, process or procedure development, documentation) 
required to use the heritage system in the new mission environment. 

Minimum Science: See Threshold Science. 

Mission Manager (MM): A D&NF program office role that provides the primary day- 
to-day interface between the program and the project. MMs have a detailed knowledge 
of the cost and schedule history, and are cognizant of the technical and programmatic 
issues for their missions. 

Mission Program Executive (PE): The Mission PEs at NASA/HQ support the 

Associate Administrator (AA) for the SMD and the Program Director in defining, 
integrating, and assessing the activities of D&NF Projects. The mission PEs are cognizant 
of project technical, schedule, and cost performance and provide course correction 
recommendations to the Program Director and directorate management. 

Principal Investigator (PI): The programmatic lead for D&NF missions. As Pi-led 
missions, the PI retains ultimate authority and responsibility for the cost, schedule, 
technical, and science performance for their mission. 

Science Floor: See Threshold Science. 

Technology Development : . 

Threshold Science: The minimum scientific requirements below which the mission is 
not considered justifiable for the proposed cost. Also referred to as Minimum Science 
and Science Floor. 
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Appendix B. Acronyms 


AA 

Associate Administrator 

AO 

Announcement of Opportunity 

CDR 

Critical Design Review 

CR 

Confirmation Review 

CSR 

Concept Study Report 

D&NF 

Discovery and New Frontiers 

DP 

Discovery Program 

DPI 

Deputy Principal Investigator 

EVM 

Earned Value Management 

FPA 

fault protection and autonomy 

FRR 

Flight Readiness Review 

HQ 

Headquarters 

IBR 

Integrated Baseline Review 

IESD 

internal electrostatic discharge 

IMS 

Integrated Master Schedule 

LCC 

life cycle cost 

LRR 

Launch Readiness Review 

MM 

Mission Manager 

MRR 

Mission Readiness Review 

MSFC 

Marshall Space Flight Center 

NASA 

National Aeronautics and Space Administration 

NFP 

New Frontiers Program 

NIAT 

NASA Integrated Action Team 

NPR 

NASA Procedural Requirements 

PCB 

Program Control Board 

PDR 

Preliminary Design Review 

PE 

Program Executive 

PI 

Principal Investigator 

PPBE 

Planning, Programming, Budgeting, and Execution 

RM 

Resource Manager 

SIR 

System Implementation Review 

SMD 

Science Mission Directorate 

SRB 

Standing Review Board 

SRR 

Systems Requirements Review 

TMCO 

technical, management, cost, and other 
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